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1 .       BACKGROUND 

1.1  DEFINITIONS 

Given  the  binary  linear  integer  programming  problem 

Minimize  { ex | Ax   b,  x  =  (0,1)}  (1) 

where  A  is  an  m  by  n  matrix,  a  surrogate  constraint  for 
problem  (1)  is  defined  as  follows: 

u(b-Ax)  >_  0,   u  =  (u,  ,u2,  .  .  .  ,u  )  ,   u    0.  (2) 

We  will  also  find  it  convenient  to  define  a  corresponding 
one-constraint  (knapsack)  surrogate  problem) : 

Minimize  {cx|u(b-Ax)    0,   u    0,   x  =  (0,1)}.     (3) 

1.2  PROPERTIES 

Because  the  vector  u  0  implies  that  the  solution  set 
to  inequality  (2)  contains  the  solution  set  to  (1) ,  we  have 
the  following  properties: 

1.  If  x*  is  a  feasible  solution  to  (1)  it  is  also 
feasible  to  (2)  and  (3) . 

2.  If  the  surrogate  constraint  (2)  has  no  feasible  solu- 
tion, then  neither  does  the  original  problem  (1) . 

1.3  USES 

Surrogate  constraints  are  used  in  conjunction  with  implicit 
enumeration  algorithms  (e.g.,  Balas)  in  several  ways.   Each 


vertex  in  an  enumeration  tree  represents  a  restriction  of  prob- 
lem (1) .   Problems  (1),  (2)  and  (3)  can  be  written  in  explicit 
terms  of  the  restriction  being  studied  by  substitution  of  the 
variables  assigned  values  by  the  restriction.   If  it  can  be 
shown  that  a  surrogate  constraint  corresponding  to  a  vertex 
has  no  feasible  solution  (completion) ,  then  the  vertex  being 
studied  can  be  fathomed  by  Property  (2) .   If  the  vertex  can- 
not be  fathomed,  Property  (1)  allows  the  surrogate  to  be 
appended  to  the  constraint  set  as  a  valid  constraint  to  be 
used  with  the  implicit  enumeration  tests  to  identify  promising 
variables  for  further  exploration.   Further,  if  the  solution 
to  Problem  (3)  corresponding  to  the  vertex  restriction  is 
known,  it  provides  an  upper  bound  on  the  original  problem  (1) 
if  it  is  feasible  for  (1)  or  a  lower  bound  on  the  vertex 
restriction  if  it  is  infeasible  for  (1)  . 

1.4   STRENGTH  OF  SURROGATES 

In  order  to  choose  u  >_  0 ,  Glover  [1965]  defined  surrogate 

1  2 

u  (b-Ax)  >  0  to  be  stronger  than  u  (b-Ax)  >  0  if 


Min{cxiu1(b-Ax)  >  0,  x=(0,l)}  >  Min{cx  j  u2  (b-Ax)  >  0 ,  x=(0,l)}, 


1    2 
for  u  ,  u   >  0.  (4) 


This  definition  states  that  if  the  corresponding  surrogate 
knapsack  problems  (3)  are  resolved,  the  surrogate  resulting 
in  the  more  restrictive  lower  bound  to  problem  (1)  is  stronger 
Essentially,  that  surrogate  eliminating  more  solutions  (as 
measured  by  the  objective  function)  is  the  stronger.   This  is 
intuitively  appealing  since  by  Property  (1)  the  surrogate 


cannot  eliminate  any  feasible  solutions  to  the  original 
problem.  Thus  by  this  definition  we  should  choose  u  as 
follows : 


Max  Min     {cx|u(b-Ax)  ^  0}  (5) 

u>0  x=(0,l) 


Unfortunately  (5)  is  difficult  to  solve  for  general  cases, 
although  Glover  [196  5]  has  studied  the  two  constraint  case. 
An  approximation  to  (5)  can  be  made  by  relaxing  the  integer 
restriction  on  x,  i.e.,  choose  u  >  0  satisfying 


Max  Min    {cx|u(b-Ax)  >_  0 }  (6) 

u>0  0<x<l 


Two  aspects  of  the  approximation  suggested  by  (6)  will  be 
investigated.   In  this  report  we  will  investigate  the 
strength  of  the  solution  to  the  approximation  (6)  as  measured 
by  definition  (4) .   In  a  separate  report  we  will  investigate 
the  speed  with  which  (6)  can  be  resolved  in  comparison  with 
alternative  approximations  to  problem  (5)  ,  Giordano  [1982]  . 

2.   THE  DUAL  MULTIPLIER  SURROGATE 

2.1   DEFINITION 

The  advantage  of  the  relaxation  to  (6)  is  that  it  can  be 
resolved  optimally  yielding  u  =  v  where  v   are  the  dual 
variables  to  the  linear  programming  (LP)  relaxation  of  (1) . 
Thus  at  any  given  vertex  restriction,  after  substituting  the 


variables  assigned  values,  the  LP  written  in  terms  of  the 

remaining  free  variables  may  be  resolved  and  the  optimal 

values  of  the  dual  variables  used  as  weights  to  form  a 

surrogate  constraint.   A  surrogate  so  formed  is  called  a 
dual  multiplier  surrogate. 

2. 2  PROPERTIES/USES 

As  with  all  surrogates,  if  it  can  be  shown  that  the  dual 
multiplier  surrogate  has  no  feasible  solution  then  the  vertex 
can  be  fathomed.   This  test  can  be  strengthened  by  requiring 
that  the  solution  to  the  surrogate  constraint  also  improve 
the  current  lower  bound  on  problem  (1),  Geoffrion  [1969]. 
In  resolving  the  corresponding  LP  for  the  dual  variables  the 
value  of  the  free  variables  may  be  used  to  solve  the  LP 
relaxation  of  (3)  directly.   Note  in  this  case  when  solving 
for  the  dual  variables  we  are  solving  the  LP  relaxation  of  (1) 
corresponding  to  the  vertex  restriction.   If  the  resulting 
values  of  the  free  variables  are  integer,  problem  (1)  has  been 
solved  for  that  vertex  and  a  new  upper  bound  on  the  original 
problem  has  been  obtained.   If  the  values  are  fractional, 
then  a  lower  bound  for  the  vertex  is  obtained.   If  desired, 
heuristics  may  be  applied  to  the  fractional  values  to  identify 
promising  variables  for  branching. 

2. 3  COMPUTATIONAL  ADVANTAGES 

The  dual  multiplier  surrogate  has  been  widely  used  in 
conjunction  with  implicit  enumeration  algorithms  and  research 


has  been  conducted  on  frequency  of  use,  maximum  number  of 
constraints  to  carry  forward,  and  related  matters.   It  is 
interesting  to  note  the  effect  of  the  use  of  the  dual 
multiplier  surrogate  on  the  problem  set  studied,  which  includes 
twenty  problems  with  up  to  5  0  variables  and  up  to  15  constraints 
Nineteen  of  the  problems  required  a  total  of  558.81  CPU  sec- 
onds (CDC  6  500)  using  a  Balas  Algorithm  with  heuristics.   The 
same  Balas  Algorithm  employing  a  dual  multiplier  surrogate 
generated  every  eight  iterations  and  carrying  a  maximum  of 
four  surrogates  forward  solved  the  nineteen,  in  3  3.78  CPU  sec- 
onds.  Problem  20,  consisting  of  50  variables  and  5  constraints, 
had  not  been  solved  optimally  after  5631  CPU  seconds  using 
the  Balas  Algorithm  but  was  solved  optimally  in  6.47  seconds 
when  the  surrogate  was  added.   The  results  are  summarized 
in  Table  1. 

2.4   OBSERVATIONS 

The  dual  multiplier  surrogate  has  been  a  very  significant 
contribution  to  implicit  enumeration.   Nevertheless,  there 
are  disadvantages  inherent  in  the  dual  multiplier  surrogate 
when  applied  to  large  problems.   A  linear  program  must  be 
solved  at  each  vertex  at  which  a  surrogate  constraint  is  to 
be  formed.   As  problems  with  larger  numbers  of  variables  are 
considered,  not  only  does  the  size  of  the  corresponding  LP's 
increase,  but  more  importantly,  the  number  of  vertices  grows 
exponentially.   Since  one  of  the  primary  advantages  of  the 
Balas  Algorithm  is  that  it  is  additive  computationally,  the 


TABLE  1 

COMPARISON  OF  SOLUTION  TIMES  USING  A  BALAS  ALGORITHM  WITH 
AND  WITHOUT  A  DUAL  MULTIPLIER  SURROGATE 
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necessity  of  solving  the  LP's  should  be  investigated.  Note 
that  the  necessity  to  solve  the  LP's  makes  the  integer  pro- 
gramming problem  more  sensitive  to  the  number  of  constraints 
than  is  otherwise  the  case.  Ideally  one  would  like  to  build 
a  surrogate  with  strength  and  computational  advantages  com- 
parable to  the  dual  multiplier  surrogate  but  requiring  less 
computation  time. 

3.   THE  CONSTRUCTION  OF  STRONGER  SURROGATES 

3.1   GEOMETRIC  INTERPRETATION 

Consider  the  minimization  problem  depicted  in  Figure  1 
consisting  of  Constraints  1  and  2  and  the  Objective  Function 
illustrated.   Surrogate  A  is  a  nonnegative  linear  combination 
of  Constraints  1  and  2.   If  the  surrogate  knapsack  problem 
consisting  of  the  Objective  Function  and  Surrogate  A  is  re- 
solved, point  2  is  the  optimal  solution.   Note  that  point  2 
is  a  lower  bound  on  the  original  problem  since  it  violates 
Constraint  2 . 

If  a  constraint  stronger  than  Surrogate  A  is  to  be  con- 
structed, then  point  2  must  become  infeasible  for  the  new 
surrogate.   This  must  be  done  such  that  point  1  remains 
infeasible.   Figure  2  depicts  such  a  surrogate.   If  the  sur- 
rogate knapsack  problem  consisting  of  the  Objective  Function 
and  Surrogate  B  is  resolved,  then  point  3  is  optimal.   Since 
point  3  satisfies  the  original  constraints,  it  is  optimal 
for  the  original  problem.   In  the  general  case  one  would  expect 


Constraint  1 


Figure  1 .   Surrogate  A 


'Constraint  1 


Figure  2.   Surrogate  B 


to  iterate  to  a  best  surrogate  without  solving  the  original 
problem.   Thus  we  seek  an  iterative  procedure  which  builds 
a  new  surrogate  for  which  the  optimal  solution  to  the  current 
surrogate  becomes  infeasible  without  admitting  less  optimal 
points . 

It  is  interesting  to  note  the  limitations  of  the  dual 
multiplier  surrogate  in  such  a  procedure.   Since  the  nonzero 
dual  variables  identify  those  constraints  which  are  binding 
at  the  optimal  solution  to  the  continuous  relaxation  of 
problem  (1),  the  dual  multiplier  surrogate  must  pass  through 
the  continuous  optimum.   Examination  of  Figure  3  reveals  that 
a  dual  multiplier  surrogate  would  not  be  able  to  eliminate 
point  1  since  it  is  constrained  to  pass  through  point  D  and 
lie  between  Limit  A  and  Limit  B  depending  on  the  specific 
values  of  the  dual  variables.   However,  a  general  surrogate 
such  as  Surrogate  C  which  is  not  constrained  to  pass  through 
point  D  can  eliminate  point  1. 

3.2   AN  ALGEBRAIC  PROCEDURE 

Let  us  define  for  the  current  surrogate: 

x*:   the  optimal  solution  to  the  current  surrogate  knapsack 

problem. 
S*:   the  slack  x*  allows  in  the  current  surrogate, 
u*:   weight  of  the  i    constraint  in  current  surrogate. 


l 


the  slack  x*  allows  in  the  i    constraint. 


Constraint  1 


Figure  3.   Limits  of  the  Dual  Multiplier  Surrogate 


Similarly,  for  the  new  surrogate  let 


u!:   The  weight  of  the  i    constraint  in  the  new  surrogate 
SI:   The  slack  x*  would  allow  in  the  new  surrogate. 


By  definition 


S0   =  I    UiSi' 

1 


and 


<5  * 


=  I 


u's*. 

1  i 


Let 


ui 


U7 


*  -. 


a .  9 
l 


(7) 


10 


where  ou  =  f(s|)  and  8  is  a  parameter  to  insure  that  the 
optimal  solution  to  the  current  surrogate  becomes  infeasible 
for  the  new  surrogate.   The  purpose  of  a.  is  to  increase  the 
weight  of  the  constraints  which  are  presently  violated.   Sub- 
stituting, we  have 

Si  =  I    (^-a.ejsj  =  S*  -  6$  ajsj. 
To  insure  x*  becomes  infeasible  for  the  new  surrogate,  choose 


9   =   (Sq/JVs*)  +  £    for  z    > 


(8) 


3.3   DISCUSSION 

We  must  insure  that  all  weights  remain  nonegative  to 
retain  the  properties  of  surrogate  constraints.   For  example, 
unless  care  is  taken,  some  weights  which  are  currently  zero 
may  become  negative  as  weights  are  changed. 

Several  possibilities  for  a.  exist;  one  is: 


1     if   s*  >  0 

l 


* 


a±  =  }    0    if  sj  ■  0. 
(  -1    if   s*  < 


Using  this  procedure,  weighting  the  new  surrogate  with  previ- 
ous surrogates  in  some  fashion  may  be  desirable  to  prevent 
abrupt  changes. 


11 


Another  choice  is 


a.   =   s*.  (10) 

i       i 


The  idea  here  is  to  increase  the  weight  of  the  violated  con- 
straints while  reducing  the  weight  of  the  satisfied  constraints 
roughly  in  proportion  to  the  degree  of  respective  violation 
or  satisfaction.   If  this  procedure  is  followed,  one  might 
consider  normalizing  the  constraints  in  some  fashion  so  that 
a  unit  of  slack  in  each  constraint  means  essentially  the  same 
thing. 

The  value  of  £  is  essentially  heuristic.   It  may  initially 
be  large  and  then  progressively  reduced  as  optimal  values  for 
the  surrogate  knapsack  problems  fail  to  improve.   A  termination 
procedure  based  on  the  number  of  such  reductions  may  be  devised. 

Finally,  if  the  new  surrogate  is  weighted  with  the  previ- 
ous surrogate,  the  size  of  the  coefficients  of  the  surrogates 
should  be  controlled  to  avoid  compromising  the  weighting 
scheme. 

4 .   EXPERIMENTATION 

4.1   METHODOLOGY 

Since  the  algebraic  procedure  suggested  above  is  heuristic 
in  nature,  a  procedure  is  needed  to  measure  the  effect  of  the 
various  schemes  and  values  for  the  parameters  inherent  in  the 
process.   If  none  of  the  optimum  solutions  to  the  current 
surrogate  knapsack  problem  are  feasible  with  respect  to  the 
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original  constraints,  then  the  optimal  solution  value  for  the 
original  problem,  if  one  exists,  must  be  greater  than  the 
optimal  solution  value  for  the  current  surrogate  knapsack 
problem  by  an  amount  at  least  as  great  as  the  least  common 
multiple  of  the  objective  function  vector.   Thus  if  the  dual 
multiplier  surrogate  is  the  initial  surrogate,  one  may  con- 
verge to  the  optimal  solution  to  problem  (1)  by  formulating 
and  solving  surrogate  knapsack  problems,  using  the  solution 
to  the  current  surrogate  as  a  lower  bound.   While  in  the 
stated  form  the  algorithm  is  not  practical  for  solving  large 
problems,  it  has  been  useful  for  experimentation  since  the 
effectiveness  of  various  schemes  for  formulating  surrogates 
can  be  measured  roughly  by  the  number  of  surrogate  knapsack 
problems  resolved  in  the  process. 

4.2   DETERMINATION  OF  a. 

The  initial  procedure  tested  was  to  form  a  surrogate  by 
simply  adding  the  constraints  violated  by  the  solution  to  the 
current  surrogate.   The  procedure  worked  reasonably  well  for 
those  problems  where  a  few  constraints  dominated.   However, 
oscillation  in  the  constraints  forming  subsequent  surrogates 
indicated  the  desirability  of  weighting  previous  surrogates 
in  order  to  include  previously  violated  constraints. 

The  next  procedures  tested  were,  given  the  current  surro- 
gate weights,  to  increase  the  weights  of  the  violated  con- 
straints and  reduce  the  weights  of  the  satisfied  constraints 
as  suggested  by  procedures  (9)  and  (10) .   Procedure  (10) 
improved  the  convergence  process  significantly. 
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Finally,  various  weighting  schemes  for  combining  the 
current  and  new  surrogates  were  tried.   Weighting  the  previ- 
ous surrogates  more  heavily  (75%)  proved  most  effective  for 
the  problem  set.   It  should  be  noted  that  various  scaling 
procedures  designed  to  prevent  the  growth  of  the  coefficients 
of  the  surrogates  while  maintaining  the  desired  proportion- 
ality were  tested  and  found  to  have  little  effect  for  the 
problem  set. 

4.3   NORMALIZATION 

Having  determined  that  constraints  should  be  weighted 
according  to  their  relative  degree  of  violation  or  satisfac- 
tion, various  normalization  procedures  were  tested.   The  idea 
was  to  avoid  weighting  a  constraint  excessively  simply  due 
to  problems  of  scale  as  can  be  seen  by  the  following  two 
constraints: 

2x  +   5x      4 
20x  +  50x      40. 

Although  the  constraints  are  equivalent,  their  slacks  differ 
in  scale  by  a  factor  of  10.   As  expected,  the  classical 
method  for  normalizing  vectors  proved  too  costly  and  caused 
excessive  rounding  errors.   Two  simple  procedures  which  were 
effective  are  as  follows: 


o        *  r  i    if  b  -  o, 

s.   =   s./d,   with   d  =   < 
1  !  lb  I   otherwise 
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0  *  ,r 

s.   ■  s./Ja.  . . 

1  1  h       11 

Linear  combinations  of  the  above  procedures  were  also  tested. 
All  methods  were  effective  without  a  clear  preference  for 
any  method.   For  the  problem  set  tested,  normalization 
was  not  a  serious  issue. 

4.4   DETERMINATION  OF  £ 

Having  determined  a  method  for  selecting  a.    and  a  normali- 
zation factor,  attention  was  directed  toward  choosing  e.   Two 
methods  which  proved  effective  were: 

8  =   S*/T  cus*  +  £    for   £  >  0. 

i 

9  =  (l+z)S*/l    ais*  +  .01    for   £  >  0. 

i 

For  the  problem  set  considered,  the  first  method  proved  better 
with  £  =  .05  initially.   The  value  of  £  was  reduced  as  surro- 
gates failed  to  be  stronger. 

5.   ALGORITHM  FOR  CONSTRUCTING  STRONGER  SURROGATES 

5.1   DISCUSSION 

Various  combinations  of  the  above  schemes  were  tested 
to  see  which  resulted  in  the  strongest  surrogates.   Each  pro- 
cedure started  with  the  dual  multiplier  surrogate  and  attempted 
to  build  stronger  surrogates.   The  process  terminated  when 
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stronger  surrogates  could  no  longer  be  constructed.   A  speci- 
fied number  of  failures  was  used  as  a  stopping  criterion. 
While  different  methods  worked  better  for  some  problems,  the 

method  chosen  set  a.  =  s.  without  normalization  and  £  =  .05 

i     1 

and  weighted  the  preceding  surrogate  75%.   If  a  successive 
surrogate  failed  to  be  stronger  than  its  predecessor,  £  was 
replaced  by  e/2  and  the  process  repeated.   If  a  surrogate 
failed  to  improve  after  three  such  e  reductions  the  process 
terminated  with  the  previous  surrogate  judged  "best". 

5.2   MEASURING  THE  STRENGTH  OF  SURROGATES 

The  strength  of  a  surrogate  is  measured  by  the  optimal 
solution  to  the  corresponding  knapsack  problem.  To  compare 
various  surrogates  the  following  measure  proved  convenient: 

1    ..i,i  i 


percent  convergence   =   |z_  —  z"j/|zn  -  z_ 


where 


z_:   objective  function  value  of  the  optimal  solution  to 
problem  (1) . 


i 


z_:   objective  function  value  of  the  optimal  solution  to 

the  continuous  relaxation  to  (1) . 
z":   objective  function  value  of  the  optimal  solution  to 

(3)  for  the  surrogate  in  question. 

The  percent  convergence  heuristically  measures  the  number  of 
infeasible  solutions  between  the  continuous  and  integer  opti- 
mum solutions  to  (1)  that  a  particular  surrogate  effectively 
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eliminates  and  is  an  indication  of  the  relative  strength  of 
two  surrogates. 

5.3   RESULTS 

Of  the  20  problems  considered,  the  solution  to  the  dual 
multiplier  surrogate  knapsack  problem  solved  the  original 
problem  directly  in  9  cases.   In  the  remaining  11  cases,  it 
was  possible  to  build  a  stronger  surrogate  in  9  cases.   The 
improvement  in  most  instances  was  substantial.   In  fact,  the 
best  surrogate  obtained  solved  an  additional  four  problems. 
The  results  are  summarized  in  Table  2. 

6.   CONCLUSIONS 

These  results  indicate  that  the  dual  multiplier  surrogate 
is  indeed  strong  as  measured  by  definition  (4) .   However, 
using  the  procedure  described  herein,  it  was  possible  to 
iterate  to  a  stronger  surrogate  in  32%  of  the  cases  attempted. 

This  suggests  further  research  to  develop  a  procedure  to 
form  surrogates  of  comparable  strength  to  the  dual  multiplier 
surrogates,  and  form  them  quickly.  Such  a  procedure  would 
avoid  using  the  dual  multiplier  surrogate  as  the  initial 
surrogate  and  avoid  resolving  each  surrogate  knapsack  problem 
for  exact  solutions  since  both  of  these  procedures  are  compu- 
tationally expensive. 


17 


TABLE  2 

CONSTRUCTION  OF  SURROGATES  STRONGER  THAN  THE 
DUAL  MULTIPLIER  SURROGATE 
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